Image processing apparatus for generating correction information, image processing method for generating correction information, and recording medium

ABSTRACT

An image processing apparatus includes an image processing unit configured to generate a first video signal for a first display apparatus, a correction unit configured to generate, when the first video signal is displayed on a second display apparatus operative to represent a brightness range narrower than a brightness range representable by the first display apparatus, correction information for correcting brightness of the video, and a recording unit configured to associate the correction information generated by the correction unit with the first video signal and hold the associated information.

BACKGROUND Field of the Disclosure

The present disclosure relates to an image processing technique forcorrecting a video image so that the video image will have suitablebrightness when displayed on a display apparatus which represents adifferent brightness range.

Description of the Related Art

In recent display apparatuses, video output performances have beenimproved. There have been developed display apparatuses that canrepresent a wide dynamic range, which is close to that of an actualsubject. For example, technique (an absolute brightness display mode)for allowing viewers to enjoy more realistic videos has been proposed.According to this technique, a subject is displayed with its actualbrightness. Namely, an imaging apparatus performs imaging whileadjusting the absolute brightness value of a subject to the brightnessvalue of the subject displayed on a display apparatus. When a displayapparatus having a wide dynamic range outputs a video captured in theabsolute brightness display mode, the image is displayed with thebrightness equivalent to that of the actual subject. Thus, the video cangive viewers more realistic feeling. Japanese Patent ApplicationLaid-Open No. 2008-187590 proposes a technique in which a user cancapture a primary subject and background with respective appropriatebrightness values by giving a single imaging instruction.

However, when a display apparatus having a dynamic range narrower thanthat of an actual subject outputs a video captured in the absolutebrightness display mode, since the video to be viewed is displayed withlower brightness, the displayed video appears dark. The displayed videoappears relatively darker than a video image captured by usingconventional automatic exposure control processing. In such cases, imageediting software can be used to check the video on a monitor and correctthe brightness. However, since a correction amount for achievingappropriate brightness differs depending on the brightness and contrastof an individual subject, editing needs to be made for each capturedscene, which is a time-consuming process. In addition, Japanese PatentApplication Laid-Open No. 2008-187590 does not consider a case in whicha video captured for displaying with absolute brightness is output on adisplay apparatus having a narrower dynamic range than that of an actualsubject.

SUMMARY

According to an aspect of the present invention, an image processingapparatus includes an image processing unit configured to generate afirst video signal for a first display apparatus, a correction unitconfigured to generate, when the first video signal is displayed on asecond display apparatus operative to represent a brightness rangenarrower than brightness range representable by the first displayapparatus, correction information for correcting brightness of thevideo, and a recording unit configured to associate the correctioninformation generated by the correction unit with the first video signaland hold the associated information.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a digitalvideo camera according to an exemplary embodiment.

FIGS. 2A and 2B are block diagrams illustrating internal configurationsof an image processing unit and a system control unit, respectively,according to the exemplary embodiment.

FIG. 3 is a flowchart illustrating image processing according to theexemplary embodiment.

FIGS. 4A and 4B illustrate input/output (I/O) characteristics of adisplay apparatus and gamma characteristics of a video, respectively,according to the exemplary embodiment.

FIG. 5A illustrates gamma characteristics of a camera according to theexemplary embodiment, and FIGS. 5B and 5C illustrate I/O characteristicsof monitors A and B, respectively, according to the exemplaryembodiment.

FIG. 6 is a flowchart illustrating processing from an exposure controlprocess to a metadata generation process according to the presentexemplary embodiment.

FIG. 7 illustrates how a subject brightness value, exposure controlvalue, and a brightness correction value change with the passage of timeaccording to the present exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, an exemplary embodiment of the present invention will bedescribed in detail. The exemplary embodiment described below is anexample for realizing the present invention. As needed, modifications onchanges could be made to the exemplary embodiment, based on aconfiguration of an apparatus to which the present invention is appliedor on various conditions. Namely, the present invention is not limitedto the following exemplary embodiment. In addition, parts of theexemplary embodiment described below may be combined as needed.

<Configuration of Apparatus>

First, a configuration and functions of a digital video camera 100according to the present exemplary embodiment will be described withreference to FIG. 1.

In FIG. 1, imaging lenses 103 are a group of lenses including a zoomlens and a focus lens and form an image of a subject. An aperture 101 isused to adjust the quantity of incident light. A neutral density (ND)filter 104 is used to adjust (reduce) the quantity of incident light ina manner different from the aperture 101. An imaging unit 122 is animaging sensor including a charge-coupled device (CCD) and acomplementary metal-oxide-semiconductor (CMOS), which converts anoptical image of a subject into an electric signal. In addition, theimaging unit 122 has various functions, such as controlling theaccumulation by an electronic shutter and changing an analog gain and areading speed. An analog-to-digital (A/D) converter 123 converts ananalog signal into a digital signal. The A/D converter 123 is used toconvert an analog signal output from the imaging unit 122 into a digitalsignal. A barrier 102 covers the imaging system including the imaginglenses 103, the aperture 101, and the imaging unit 122 of the digitalvideo camera 100 (which will hereinafter be referred to as a camera100), to prevent soiling of and damage to the imaging system.

An image processing unit 124 performs color conversion processing, gammacorrection processing, digital gain addition processing, and the like ondata from the A/C converter 123 or data from a memory control unit 115.In addition, the image processing unit 124 performs predeterminedcalculation processing by using captured image data. Based on thecalculation result, a system control unit 150 performs exposure controlprocessing, ranging control processing, white balance (WB) controlprocessing, and the like. Accordingly, for example, autofocus (AF)processing, automatic exposure (AE) processing, and automatic whitebalancing (AWE) processing based on a through-the-lens (TTL) method areperformed. The image processing unit 124 will be described in detailbelow.

Output data from the A/D converter 123 is directly written into a memory132 via the image processing unit 124 and the memory control unit 115 orvia the memory control unit 115. Image data is captured by the imagingunit 122 and is converted into digital data by the A/D converter 123.The memory 132 holds the digital data. In addition, the memory 132 holdsimage data to be displayed on a display unit 128. The memory 132 has asufficient storage capacity for storing moving images and sound for apredetermined time.

In addition, the memory 132 serves as a memory (a video memory) forimage display. A digital-to-analog (D/A) converter 113 converts imagedisplay data stored in the memory 132 into an analog signal and suppliesthe analog signal to the display unit 128. In this way, the image datawritten in the memory 132 is displayed by the display unit 128 via theD/A converter 113. The display unit 128 displays an image on a displaydevice such as a liquid crystal display (LCD) based on an analog signalfrom the D/A converter 113. More specifically, the A/C converter 123converts analog signals into digital signals, and the D/A converter 113converts the digital signals accumulated in the memory 132 into analogsignals and sequentially forwards the converted analog signals to thedisplay unit 128. The display unit 128 sequentially displays the analogsignals. In this way, the display unit 128 can function as an electronicview finder (EVF) and display live view images.

A non-volatile memory 156 is an electrically erasable and recordablememory. For example, an electrically erasable programmable read-onlymemory (EEPROM) is used as the non-volatile memory 156. The non-volatilememory 156 holds constants, programs, etc. for operations of the systemcontrol unit 150. These programs are used for executing variousflowcharts, which will be described below in the present exemplaryembodiment.

The system control unit 150 comprehensively controls the camera 100. Byexecuting the programs stored in the non-volatile memory 156, the systemcontrol unit 150 realizes the following processing of the presentexemplary embodiment. A random access memory (RAM) is used as a systemmemory 152. For example, the system control unit 150 loads constants,variables, and programs for operations of the system control unit 150from the non-volatile memory 156 and expands the loaded constants,variables, and programs in the system memory 152. In addition, bycontrolling the memory 132, the D/A converter 113, the display unit 128,etc., the system control unit 150 performs display control processing.

A system timer 153 is a timer that measures time used for various kindsof control processing and time of an individual internal clock.

A mode selection switch 160, a video recording switch 161, and anoperation unit 170 are operation units for inputting various operationinstructions to the system control unit 150.

The mode selection switch 160 switches the operation mode of the systemcontrol unit 150 to any one of a moving image recording mode, a stillimage recording mode, a playback mode, etc. The moving image recordingmode and the still image recording mode include an automatic shootingmode, an automatic scene determination mode, a manual mode, variousscene modes each of which is set depending on the scene to be captured,a program AE mode, and a custom mode. By operating the mode selectionswitch 160, the user can directly switch the current mode to any one ofthese modes included in the moving image recording mode. Alternatively,after the user switches to the mode to the moving image recording modeby operating the mode selection switch 160, the user may use a differentoperation component to switch the mode to any one of these modesincluded in the moving image recording mode. By operating the videorecording switch 161, the user can switch between a stand-by state andan imaging state. When the user turns on the video recording switch 161,the system control unit 150 starts a series of operations from reading asignal sent from the imaging unit 122 to writing video recording data toa recording medium 190.

By selecting various functional icons displayed on the display unit 128,the operation components of the operation unit 170 are assigned withfunctions appropriate to the individual scenes and serve as variousfunction buttons. Examples of the function buttons include an endbutton, a back button, an image scrolling button, a jump button, anarrow-down button, and an attribute change button. For example, whenthe user presses a menu button, the display unit 128 displays a menuscreen on which various settings can be made. The user can intuitivelymake various settings by using the menu screen displayed on the displayunit 128, a four-direction (left, right, top, and bottom) button, and aSET button.

A power supply control unit 180 includes a battery detection circuit, adirect current (DC)-DC converter, and a switch circuit for switchingblocks to be energized and detects whether a battery is mounted, thetype of the battery, and the remaining battery capacity. In addition,the power supply control unit 180 controls the DC-DC converter based onthe detection result and an instruction from the system control unit 150and supplies a necessary voltage to various units including therecording medium 190 for a necessary period of time.

A power supply unit 130 includes a primary battery such as an alkalinebattery or a lithium battery, a secondary battery such as anickel-cadmium (NiCd) battery, a nickel-metal hydride (NiMH) battery ora lithium (Li) ion battery, an alternating current (AC) adaptor, etc. Aninterface (I/F) 118 is connected to the recording medium 190 such as amemory card or a hard disk or to an external apparatus. In FIG. 1, theI/F 118 is connected to the recording medium 190. The recording medium190 is a medium such as a memory card on which captured images arerecorded. The recording medium 190 may include a semiconductor memory,or a magnetic disk.

<Internal Configuration of Image Processing Unit>

Next, an internal configuration of the image processing unit 124according to the present exemplary embodiment will be described withreference to FIG. 2A.

FIG. 2A illustrates peripheral units of the image processing unit 124and an internal configuration thereof. Description of the WB controlprocessing and sharpness control processing normally performed by theimage processing unit 124 will be avoided. Functional blocks included inthe image processing unit 124 can acquire various kinds of data used inthe camera 100, including elements needed for exposure controlprocessing (exposure parameters), such as the aperture, ND information(information about the density of the ND filter, etc.), InternationalStandards Organization (ISO) sensitivity, gain, and shutter speed viathe system control unit 150.

In FIG. 2A, the detection unit 201 detects data from the A/D converter123 or data from the memory control unit 115 and obtains arepresentative value of the data. A brightness value (BV) calculationunit 202 calculates a BV (an absolute brightness value) from a referencesignal, the aperture, the ISO sensitivity, the gain, the shutter speed,and the representative value acquired by the detection unit 201.

An absolute-brightness-code determination unit 203 acquires input/output(I/O) characteristics of a display apparatus from the non-volatilememory 156 or the memory control unit 115. Next, theabsolute-brightness-code determination unit 203 determines an absolutebrightness code such that the brightness value (output brightness)output by the display apparatus matches the BV (the absolute brightnessvalue) calculated by the BV calculation unit 202 based on the I/Ocharacteristics of the display apparatus. The I/O characteristics of thedisplay apparatus may previously be stored in the non-volatile memory156 or may be input by the user. Alternatively, the I/O characteristicsmay be acquired from the display apparatus being connected. Theacquisition method is not particularly limited.

A gamma correction unit 205 performs gamma correction on data from theA/D converter 123 or data from the memory control unit 115.

<Internal Configuration of System Control Unit>

Next, an internal configuration of the system control unit 150 accordingto the present exemplary embodiment will be described with reference toFIG. 23.

FIG. 23 illustrates peripheral units of the system control unit 150 andan internal configuration thereof. A first exposure calculation unit 211acquires gamma characteristics from the gamma correction unit 205 andacquires a determined absolute brightness code from theabsolute-brightness-code generation unit 203. Based on the acquiredinformation, the first exposure calculation unit 211 determines a firstexposure control amount for outputting a video with absolute brightnessfrom the display apparatus. The system control unit 150 performsexposure control processing based on the determined first exposurecontrol amount.

Based on a target brightness value of the face of a person, the averagebrightness of the entire imaging plane, and the like stored in thenon-volatile memory 156, a second exposure calculation unit 212calculates an exposure control amount so that the brightness of arelevant subject falls within a predetermined target range.

A brightness-correction-value generation unit 213 generates a brightnesscorrection value, which is used as correction information for correctingthe brightness of a video so that the video will have brightnesssuitably displayed by a display apparatus having a brightness rangedifferent from what assumed when the camera 100 has captured the video.A metadata generation unit 214 generates recorded data from thebrightness correction value generated by the brightness-correction-valuegeneration unit 213. A signal superimposition unit 215 superimposes therecorded data generated by the metadata generation unit 214 onto a videosignal processed by the image processing unit 124.

<Image Processing>

Next, image processing according to the present exemplary embodimentwill be described.

The following example will be described assuming that the videooutputting display apparatus is a display apparatus (a monitor A) havinga wide dynamic range that can represent a first brightness range, whichis close to the brightness of a subject, and that a video is captured inthe absolute brightness display mode. The following example also assumesthat a viewer watches the video on a display apparatus (a monitor B)having a dynamic range that can represent a second brightness range,which is narrower than the first brightness range. Thus, a brightnesscorrection value is generated to correct the brightness of the videocaptured in the absolute brightness display mode so that the video willbe displayed with brightness equivalent to that of a video captured inconventional automatic exposure control processing. The generatedbrightness correction value is associated and stored as metadata.

In the conventional automatic exposure control processing, exposureparameters are adjusted so that the output brightness of a relevantsubject falls within a predetermined target range. For example, exposurecontrol processing is performed so that, for example, the face of aperson is continuously displayed with approximately 70% of the maximumoutput brightness representable by the display apparatus. By performingthis control processing, even if the display apparatus has a narrowdynamic range, the output brightness of the face of the person canalways be displayed with suitable brightness for the display apparatusof the narrow dynamic range. Hereinafter, a mode in which a cameracaptures and stores a video such that the output brightness of therelevant subject maintains at a brightness level suitable for anotherdisplay apparatus having a different brightness range will be referredto as a relative brightness display mode.

Next, image processing performed in the absolute brightness display modeaccording to the present exemplary embodiment will be described withreference to FIG. 3.

The processing in FIG. 3 is realized when the system control unit 150expands a program stored in the non-volatile memory 156 in the systemmemory 152 and executes the program.

In step S301, the system control unit 150 causes the detection unit 201to detect data from the A/D converter 123 or data from the memorycontrol unit 115. By performing this detection, the detection unit 201obtains a representative value of the data. The detection unit 201obtains the representative value, for example, calculating the averagebrightness of a center portion, calculating the average brightness of acertain subject such as a face, or acquiring the brightness of adesignated point. The detection unit 201 may obtain the representativevalue by using a different method. Selecting a detection target requirescaution. Namely, since the dynamic range of the camera 100 could changewhen exposure control processing is performed later, it could be betterto select a detection target that can be detected even after the dynamicrange is changed. For example, thresholds for upper and lower limits maybe set based on the dynamic range of the camera 100, and a target thatfalls between the thresholds may be determined as a selection target.

In step S302, the system control unit 150 causes the BV calculation unit202 to calculate a BV (an absolute brightness value) of therepresentative value. For example, the BV may be calculated from thereference signal, the aperture, the ISO sensitivity, the gain, theshutter speed, and the representative value acquired by the detectionunit 201. The BV of a reference signal, i.e., a reference BV, in theAdditive System of Photographic Exposure (APEX) expression is obtainedby expression 1.Reference BV=2^((Av+Tv−Sv))×(0.32×k)[cd/m²]  (Expression 1)

Expression 1 includes an aperture value Av, a shutter speed Tv, and anexposure control value Sv obtained by converting imaging sensitivityinto an APEX unit. Expression 1 also includes a calibration coefficientk. To obtain an input of 18% gray, the calibration coefficient k is usedwhen a brightness value expressed in an APEX unit is converted intocd/m² (or nit), which is the unit of the absolute brightness. In thepresent exemplary embodiment, the calibration coefficient k is set to12.5. A brightness value Z expressed in the APEX unit can be convertedinto an absolute brightness value X, based on a relational expressionlog₂(X/0.32×k)=Z. Namely, the absolute brightness value X is calculatedby 2^(z)×(0.32×k). For example, when Av=F4.0, Tv=1/128, Sv=ISOsensitivity 200, the reference BV is calculated by expression 1.Reference BV=2^(4(Av)+7(Tv)−6(Sv)))×(0.32×12.5)=128 [cd/m²]A reference signal code is calculated by expression 2.Reference signal code=(2^(bit number))×(reference brightness value[%]/dynamic range [%])  (Expression 2)When the dynamic range of the camera 100 is 1200%, the ratio of thereference brightness value to the upper limit of the brightness value is20%, and the bit number of the data is 14, the reference signal code canbe calculated as follows:Reference signal code=(2¹⁴)×(20/1200)=273.

When the representative value code is 2132 and the absolute brightnessvalue X is the difference between the BV of the representative value andthe reference BV, the following equation is established.Representative value code=Reference signal code×(2^(X))   (Expression 3)

By substituting the above values into expression 3, the followingequation is established.2132=273×(2^(X))X=2.96Thus, the BV of the representative value is calculated as follows:BV of the representative value=2^(2.96)×128 [cd/m²]=1000 [cd/m²]

The above calculation method is merely an example, and the presentexemplary embodiment is not limited thereto. A different method may beused to obtain the BV of the representative value. Further, the BB maybe calculated from a signal acquired from an external sensor or thelike.

In step S303, the system control unit 150 causes theabsolute-brightness-code generation unit 203 to determine an absolutebrightness code such that the output brightness matches the BV of therepresentative value. For example, the absolute brightness code iscalculated based on the I/O characteristics of the display apparatus andthe BY of the representative value calculated in step S302. FIG. 4Aillustrates a relationship between the BV of the representative valueand the I/O characteristics of the display apparatus. In the case of theI/O characteristics of the display apparatus illustrated in FIG. 4A, theabsolute-brightness-code generation unit 203 determines a unique inputcode such that the output brightness matches the BV of therepresentative value. The absolute brightness code may be obtained fromthe function as in FIG. 4A or may be calculated by an expression of theI/O characteristics and the BV of the representative value.Alternatively, an I/O relationship may be held as table data, and theabsolute brightness code may be selected from the table data based onthe BV of the representative value. The determination method accordingto the present exemplary embodiment is not limited to these methods.Namely, a different method may be applicable.

In step S304, the first exposure calculation unit 211 in the systemcontrol unit 150 calculates a signal conversion coefficient based on theabsolute brightness code determined in step S303. The first exposurecalculation unit 211 calculates an exposure control amount to match theoutput from the gamma correction unit 205 with the absolute brightnesscode for the representative value. For example, the first exposurecalculation unit 211 acquires gamma characteristics from the gammacorrection unit 205 and calculates a first exposure control amount basedon the gamma characteristics. FIG. 4B illustrates the gammacharacteristics. According to these gamma characteristics, when theinput code is Y, the output code matches with the absolute brightnesscode. If the gamma characteristics are inverse gamma characteristicswith respect to the I/O characteristics of the display apparatus andwhen the data is a representative value, an exposure control amount formatching the output code with the absolute brightness code is calculatedby expression 4.Exposure control amount=Y/representative value   (Expression 4)

In the example described in the present exemplary embodiment, the gammacharacteristics have the inverse gamma characteristics. However, thepresent exemplary embodiment is not limited to such example. The gammacharacteristics may have different characteristics.

In step S305, the system control unit 150 performs exposure controlprocessing based on the exposure control amount calculated by the firstexposure calculation unit 211. For example, if the exposure controlamount is obtained by expression 4, the system control unit 150 performsexposure control processing by adding or subtracting the exposurecontrol value to or from the current exposure control amount. As aspecific example, if the exposure control amount is ½, the systemcontrol unit 150 closes the aperture by one step. The exposure controlmethod may be changed depending on the exposure control amount. Forexample, if the exposure control amount is so large that the exposure isgreatly changed, the exposure control value may be changed graduallyover time, rather than changing the exposure control value at one time.In this way, the video could be displayed smoothly. In addition, in thepresent exemplary embodiment, the exposure parameters for calculatingthe exposure control amount are not particularly limited. The aperturesetting, the ND information, the ISO sensitivity, the gain, or theshutter speed may be changed. Some of these values may be changed at thesame time.

In step 306, the system control unit 150 causes the gamma correctionunit 205 to perform gamma correction on data from the A/D converter 123or data from the memory control unit 115.

By performing the above processing, the representative value of thetarget data is output from the camera 100 as the absolute brightnesscode, and by inputting the absolute brightness code to the displayapparatus, the display apparatus displays the video with the BV of therepresentative value. Namely, the display apparatus can display thesubject with the absolute brightness value. Thus, the user can enjoyvideo with realistic sensation.

Next, with reference to FIG. 3, image processing in the relativebrightness display mode will be described in comparison with the imageprocessing in the absolute brightness display mode.

In step S301, the system control unit 150 performs detection as in theabsolute brightness display mode. A representative value may be obtainedin the same way as in the absolute brightness display mode.Alternatively, a different subject or a different method may be used.

In step S302, the system control unit 150 calculates a BV of therepresentative value. If the first exposure calculation unit 211 doesnot use the BV later, the processing in step S302 may be omitted.

Since the absolute brightness code generation processing in step S303 isnot needed in the relative brightness display mode, this processing isnot carried out.

In step S304, the system control unit 150 calculates an exposure controlamount. In the relative brightness display mode, the system control unit150 calculates an exposure control amount so that the brightness of acertain subject reaches a target value previously determined based onthe representative value. As an example, a case will be described inwhich the representative value is the brightness of a face and thetarget value displays the face at 70% of the output brightness. In thiscase, assuming that the gamma characteristics are inverse gammacharacteristics of the I/O characteristics of the display apparatus andthat the bit number of the data is 14, the exposure control amount iscalculated by expression 5.Exposure control amount=(70%*(2¹⁴))/representative value  (Expression 5)

In the above example, the target value is directly related to therepresentative value. However, the target value may be a valuecalculated from the representative value, for example, may be a valuerelated to the BV. The relationship between the representative value andthe target value is not particularly limited.

In step S305, the system control unit 150 performs exposure controlprocessing based on the exposure control amount calculated by the firstexposure calculation unit 211.

Step S306 is carried out as in the absolute brightness display mode.

Thus, the relative brightness display mode differs from the absolutebrightness display mode in that the absolute-brightness-codedetermination processing is not performed and that the first exposurecalculation unit 211 does not calculate the exposure control amountbased on the absolute brightness code.

In the present exemplary embodiment, while the representative value isthe brightness of a face, the representative value is not limitedthereto. The average brightness value of the entire imaging plane may beused. Alternatively, a brightness evaluation value obtained bycenter-weighted photometry in which the weight near the center in theimaging area is increased may be used. In addition, an arbitrary valuemay be set as the target value, depending on the subject or imagingconditions.

In the above example according to the present exemplary embodiment, tofacilitate understanding of the difference between the relativebrightness display mode and the absolute brightness display mode, thefirst exposure calculation unit 211 also used in the relative brightnessdisplay mode. According to the present exemplary embodiment, in theabsolute brightness display mode, the first exposure calculation unit211 calculates an exposure control amount, and the exposure controlprocessing performed when a video is captured is carried out by usingthe exposure control value calculated by the first exposure calculationunit 211 or an exposure control value specified by a user operation. Inaddition, in the relative brightness display mode, the second exposurecalculation unit 212 calculates an exposure control amount. When a videois captured, the second exposure calculation unit 212 calculates anexposure control amount by acquiring a detection value from thedetection unit 201 and a target value of the brightness of a relevantsubject from the non-volatile memory 156.

Next, a case will be described in which a video captured in the absolutebrightness display mode is output to the monitors A and B, which havedifferent dynamic ranges, without applying any correction.

The description will be made based on the following conditions. Themonitor A has a maximum output brightness of 10000 cd/m², and the bitnumber of the data is 12. The monitor B has a maximum output brightnessof 100 cd/m², and the bit number of the data is 8. The bit number of thedata of the camera gamma is 12. The I/O characteristics of the monitor Aare inverse gamma characteristics of the camera gamma, and the I/Ocharacteristics of the monitor B is γ2.2. A subject is the face of aperson, and the absolute brightness value of the subject is 100 cd/m².FIG. 5A illustrates the gamma characteristics of the camera, and FIGS.5B and 5C illustrate the I/O characteristics of the monitors A and B.More specifically, FIG. 5A illustrates gamma characteristics 601 of thecamera. An input code A is an input code corresponding to when a face of100 cd/m² is captured. FIGS. 5B and 5C illustrate the I/Ocharacteristics 602 and 603 of the monitors A and B, respectively.

In the above absolute brightness display mode, the exposure processingof the camera and the video signal are controlled so that the videosignal of the captured face area matches with the input code A of thecamera gamma. In this way, the face area will be output with thebrightness of 100 cd/m² on the monitor A.

First, the video output by the monitor A will be described. The inputcode A has been converted into an output code 2048 based on the cameragamma 601, and the output code 208 is stored. When the converted codevalue 2048 is input to the monitor A, the code value 2048 is convertedbased on the I/O characteristics 602 of the monitor A and output with100 cd/m².

Next, the video output by the monitor B will be described. The inputcode A has been converted into the output code 2048 based on the cameragamma 601, and the output code 2048 is stored. When the converted codevalue 2048 is input to the monitor B, first, to convert the 12-bitsignal into an 8-bit signal, the code value 2048 is divided by 16, andas a result, a code value 128 is obtained. The input code 128 isconverted based on the I/O characteristics 603 of the monitor B andoutput with approximately 15 cd/m².

In this way, in the case of the monitor A, the video captured in theabsolute brightness display mode is output to the monitor A in a statein which the absolute brightness value of the subject matches thebrightness value of the subject displayed on the monitor A. However, inthe case of the monitor B, the video is output to the monitor B withapproximately 15 cd/m², which is rather dark. If the video is capturedin the relative brightness display mode, the exposure is controlled toachieve 70% of the output brightness of the monitor B. Thus, theperson's face is output with 70 cd/m² on the monitor B. Namely, thevideo appears darker than that displayed by using conventional automaticexposure control processing.

<Processing from Exposure Control Processing to Metadata GenerationProcessing>

Next, processing from exposure control processing to metadata generationprocessing according to the present exemplary embodiment will bedescribed with reference to FIGS. 6 and 7.

FIG. 7 illustrates change of a subject brightness value, exposurecontrol amounts, and a brightness correction value over time. FIG. 7illustrates an average brightness value 701 of a face area as an imagingtarget subject, an exposure control amount 702 in the absolutebrightness display mode, and an exposure control value 703 in therelative brightness control mode. FIG. 7 also illustrates a brightnesscorrection value 704 for correcting brightness. When a video, which hasbeen captured in the absolute brightness display mode to be displayed onthe monitor A, is displayed on the monitor B, the brightness of thevideo is decreased on the monitor B. The brightness correction value 704is used in such cases.

Next, as an example, a case will be described in which the brightness ofa face as a subject changes over time as indicated by the averagebrightness value 701 in FIG. 7. In this example, the subject appearsbright at timing T1, becomes darker at timing T2, and returns to theinitial brightness at timing T3.

In step S601, the system control unit 150 determines whether a user hasgiven an exposure instruction. The determination is made based onwhether the user has manually operated the operation unit 170 and set acertain exposure level. If the user has specified any exposure setting,a corresponding exposure control value is added to the setting. If theuser has given an exposure instruction (YES in step S601), theprocessing proceeds to step S603. If not (NO in step S601), theprocessing proceeds to S602.

In step S602, the first exposure calculation unit 211 of the systemcontrol unit 150 calculates a first exposure control amount, which is anexposure control value in the absolute brightness display mode. Next,the processing proceeds to step S603.

In step S603, the system control unit 150 performs the exposure controlprocessing in the absolute brightness display mode. If the processing iscarried out via step S602, the exposure control processing is performedusing the first exposure control value. In the present exemplaryembodiment, since the imaging is performed in the absolute brightnesscontrol mode, the first exposure control value is used during imaging.In addition, the exposure control amount is a sum of APEX values. Whenthe aperture is F7.0, the shutter speed is 1/128, the ISO sensitivity isISO 200, and the ND filter is off, the exposure control amount isobtained as follows:11(F7.0)+7(1/128)−6(ISO 200)+0(ND Off)=12

In addition, as indicated by the exposure control amount 702, theexposure control amount in the absolute brightness display mode does notchange while the brightness of the subject changes. In other words, thebrightness of the subject is directly output to the monitor withoutchanging the exposure control amount. Namely, the brightness value ofthe subject output to the monitor increases or decreases with the changeof the brightness value of the subject. In the example in the presentexemplary embodiment, the exposure is not changed. However, the presentexemplary embodiment is not limited to this example. The exposure may bechanged in view of gradation characteristics and the like.

In step S604, the system control unit 150 acquires an imaging exposurecontrol amount, which is an exposure control amount during imaging.Detailed description of this processing will be avoided.

In step S605, the second exposure calculation unit 212 of the systemcontrol unit 150 calculates a second exposure control amount, which isan exposure control value in the relative brightness display mode. Whilethe exposure control in the absolute brightness display mode is beingperformed, the brightness of the video image is detected, and the secondexposure calculation unit 212 calculates a second exposure controlamount in the relative brightness display mode. More specifically, thesecond exposure control amount is calculated such that the brightness ofthe face is maintained at approximately 70% of the output brightness ofthe monitor B. As indicated by the exposure control value 703, thesecond exposure control amount is changed according to the change amountof the brightness of the subject.

In step S606, the brightness-correction-value generation unit 213 of thesystem control unit 150 calculates the difference value between theimaging exposure control amount and the second exposure control amountand generates a brightness correction value based on the calculationresult. In the present exemplary embodiment, since the first exposurecontrol amount, which is an exposure control value in the absolutebrightness display mode, has been used during imaging, the differencevalue is the difference between the first exposure control amount 702and the second exposure control amount 703. The difference is thedifference between the sums of the APEX values. Thus, the brightnesscorrection value has characteristics as indicated by the brightnesscorrection value 701 and increases and decreases over time. Thebrightness correction value is not limited to the difference between theexposure control amounts. A different brightness correction value may beused as long as the brightness correction value can correct thedifference in the brightness values of videos caused by the differencebetween the output brightness ranges of the monitors.

In step S607, the metadata generation unit 214 of the system controlunit 150 generates the brightness correction value generated by thebrightness-correction-value generation unit 213 as metadata. Next, thesignal superimposition unit 215 superimposes the metadata onto thevideo. A video signal on which the metadata has been superimposed isrecorded in the recording medium 190.

<Brightness Correction Processing>

Next, brightness correction processing will be described with referenceto FIG. 7. In this processing, the brightness of a video captured in theabsolute brightness display mode is corrected based on the metadatastored in a video signal so that the video will have brightness to besuitably displayed by the monitor B.

Since the brightness correction value up to the timing T1 is level 4,the brightness is corrected by multiplying the video signal by a 16times gain. Likewise, since the brightness correction value in the timebetween the timing T1 and the timing T2 is level 3, the brightness iscorrected by multiplying the video signal by an 8 times gain. Likewise,since the brightness correction value in the time between the timing T2and the timing T3 is level 5, the brightness is corrected by multiplyingthe video signal by a 32 times gain. Likewise, since the brightnesscorrection value in the time after the timing T3 is level 4, thebrightness is corrected by multiplying the video signal by a 16 timesgain. The correction processing for multiplying the corresponding gainsmay be performed by the camera or the display apparatus. In addition,the correction processing may be performed by changing a lookup table(LUT) or gamma correction.

In this way, the present image processing apparatus can correct thebrightness of a video captured in the absolute brightness display modefor display on the monitor A such that the video image will havebrightness to be suitably displayed on the monitor B having a dynamicrange narrower than that of the monitor A. Thus, for example, even whenthe maximum output brightness of the monitor B is 100 cd/m², since thevideo is displayed at 70 cd/m², which is 70% of the maximum outputbrightness, the face displayed does not appear dark. Accordingly, evenwhen the brightness of the face changes due to change in brightness of alight source that illuminates the subject, this image processingapparatus can easily convert the brightness into suitable brightnessthat maintains the brightness value of the face displayed on the monitorat 70 cd/m².

In addition, in the relative brightness display mode, to avoid abruptbrightness change when the brightness is adjusted to a predeterminedtarget value, responsiveness of the exposure control processing isadaptively changed according to an individual shooting scene. Morespecifically, an upper limit may be set to a step amount, which is anexposure control change amount per unit time. Alternatively, the stepamount is changed according to the degree of deviation from the targetvalue. For this purpose, information about the exposure controlresponsiveness may be generated as a brightness correction value, andthe generated information may be associated and stored as metadata. Thevideo brightness corrected for display on the monitor B may be adjustedbased on this exposure control responsiveness.

In addition, in the relative brightness display mode, overexposure of avideo can be reduced by changing the dynamic range based on the contrastand output brightness of an individual captured scene. Morespecifically, by changing a knee point of the gamma characteristics, ahigh-brightness area is compressed, and the dynamic range is expanded.Dynamic range information for display on the monitor B may be generatedas a brightness correction value and stored as metadata. When thebrightness of a video is corrected for display on the monitor B, thehigh-brightness area is compressed based on the dynamic rangeinformation to reduce overexposure. To avoid abrupt brightness change,as with the exposure control processing, responsiveness may be given tothe dynamic range control processing. With respect to responsiveness ofthe dynamic range control processing, a brightness correction value maybe generated and stored as metadata, and responsiveness ofhigh-brightness compression performed on a video which is to becorrected for display on the monitor B may be changed.

The present exemplary embodiment has been described based on an examplewhich calculates an exposure control value of a video captured in theabsolute brightness display mode such that the video can be suitablydisplayed in the relative brightness display mode on a monitor having anarrower dynamic range. However, the present invention is not limited tothis example. The present invention is applicable to any monitor havinga different representable brightness range.

The present exemplary embodiment has been described assuming that theexposure control processing for display on the monitor A is performed inthe absolute brightness display mode and the exposure control processingfor display on the monitor B is performed in the relative brightnessdisplay mode. However, the exposure control method for display on amonitor is not limited to the above example. The exposure control may beperformed in such a manner in which both the monitors A and B operate inthe absolute brightness display mode or in which both the monitors A andB operate in the relative brightness display mode.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit. (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

According to the above described exemplary embodiments, it is possibleto correct brightness of a video image for a display apparatus which canmake a representation in a wide brightness range, such that thebrightness becomes suitable for a display apparatus which makes arepresentation in a narrower range.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-208035, filed Oct. 24, 2016, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: atleast one processor configured to function as: an image processing unitconfigured to generate a first video signal for a first displayapparatus; a first exposure calculation unit configured to calculate afirst exposure control value when a video is captured for the firstdisplay apparatus; a second exposure calculation unit configured tocalculate a second exposure control value when the video is captured fora second display apparatus, which is a different apparatus from thefirst display apparatus and operative to represent a dynamic rangenarrower than a dynamic range represented by the first displayapparatus; a correction unit configured to generate, based on the firstexposure control value and the second exposure control value, correctioninformation for correcting brightness of the video when the first videosignal is displayed on the second display apparatus; and a recordingunit configured to associate the correction information generated by thecorrection unit with the first video signal and hold the associatedinformation.
 2. The image processing apparatus according to claim 1,wherein the at least one processor is further configured to function asfollowing units: an imaging unit configured to capture an image andgenerate a video signal; and an exposure control unit configured tocontrol exposure at an imaging time; wherein the exposure control unitperforms exposure control processing at the imaging time based on thefirst exposure control value calculated by the first exposurecalculation unit.
 3. The image processing apparatus according to claim2, wherein the first exposure calculation unit calculates the firstexposure control value such that a brightness value of a relevantsubject output on the first display apparatus is equivalent to anabsolute brightness value of the relevant subject.
 4. The imageprocessing apparatus according to claim 2, wherein the first exposurecalculation unit calculates the first exposure control value such thatthe brightness value of the relevant subject output on the first displayapparatus falls within a predetermined target range.
 5. The imageprocessing apparatus according to claim 2, wherein the correction unitgenerates the correction information based on a difference between thefirst exposure control value and the second exposure control value. 6.The image processing apparatus according to claim 5, wherein the secondexposure calculation unit calculates the second exposure control valuesuch that a brightness value of the relevant subject output on thesecond display apparatus is equivalent to an absolute brightness valueof the relevant subject.
 7. The image processing apparatus according toclaim 5, wherein the second exposure calculation unit calculates thesecond exposure control value such that the brightness value of therelevant subject output on the second display apparatus falls within apredetermined target range.
 8. The image processing apparatus accordingto claim 1, wherein the correction unit generates dynamic rangeinformation for display on the second display apparatus as thecorrection information.
 9. The image processing apparatus according toclaim 1, wherein the correction unit generates information aboutresponsiveness of change to a video signal in the dynamic rangerepresented on the second display apparatus as the correctioninformation.
 10. The image processing apparatus according to claim 9,wherein the information about the responsiveness is a change amount ofexposure control processing adaptively changed according to anindividual captured scene or a change amount of dynamic range controlprocessing.
 11. The image processing apparatus according to claim 2,wherein the exposure control processing is performed by using at leastone of an aperture, neutral density information, a sensitivity, a gain,and a shutter speed.
 12. An image processing method comprising:performing image processing in which a first video signal is generatedfor a first display apparatus; a first exposure calculating in which afirst exposure control value when a video is captured for the firstdisplay apparatus is calculated; a second exposure calculating in whicha second exposure control value when the video is capture for the seconddisplay apparatus is calculated, the second display apparatus being adifferent apparatus from the first display apparatus and operative torepresent a dynamic range narrower than a dynamic range representable bythe first display apparatus; performing correction in which correctioninformation for correcting brightness of a video when the first videosignal is displayed on the second display apparatus is generated; andassociating the generated correction information with the first videosignal and storing the associated information.
 13. A non-transitorycomputer-readable storage medium holding a program that causes acomputer to function the following steps: performing image processing inwhich a first video signal is generated for a first display apparatus; afirst exposure calculating in which a first exposure control value whena video is captured for the first display apparatus is calculated; asecond exposure calculating in which a second exposure control valuewhen the video is captured for the second display apparatus iscalculated, the second display apparatus being a different apparatusfrom the first display apparatus and operative to represent a dynamicrange narrower than a dynamic range representable by the first displayapparatus; performing correction in which correction information forcorrecting brightness of a video when the first video signal displayedon the second display apparatus is generated; and associating thegenerated correction information generated in the correction with thefirst video signal and storing the associated information.